Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Go dependencies #12215

Merged
merged 6 commits into from
Feb 3, 2023

Conversation

dbussink
Copy link
Contributor

@dbussink dbussink commented Feb 2, 2023

This updates the Go dependencies to the latest versions before the Vitess 16 freeze so we can start with an up to date list.

Also updates vitess-mixin since Dependabot is reporting a bunch of things on that.

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

@vitess-bot vitess-bot bot added NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Feb 2, 2023
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Feb 2, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@dbussink dbussink removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Feb 2, 2023
This updates the Go dependencies to the latest versions before the
Vitess 16 freeze so we can start with an up to date list.

Also updates vitess-mixin since Dependabot is reporting a bunch of
things on that.

Signed-off-by: Dirkjan Bussink <[email protected]>
@dbussink dbussink force-pushed the dbussink/update-go-dependencies branch from 2cd5aae to d39c1fc Compare February 2, 2023 15:35
@frouioui frouioui mentioned this pull request Feb 2, 2023
35 tasks
@frouioui
Copy link
Member

frouioui commented Feb 2, 2023

Thank you @dbussink 🙇🏻

This is now available using generics.

Signed-off-by: Dirkjan Bussink <[email protected]>
@frouioui frouioui merged commit e596c1d into vitessio:main Feb 3, 2023
@frouioui frouioui deleted the dbussink/update-go-dependencies branch February 3, 2023 00:28
frouioui added a commit to planetscale/vitess that referenced this pull request Nov 21, 2023
* Remove replication manager and run VTOrc in all e2e tests (#12149)

* feat: remove replication manager

Signed-off-by: Manan Gupta <[email protected]>

* test: run VTOrc in all e2e tests (some exceptions)

Signed-off-by: Manan Gupta <[email protected]>

* docs: add summary notes changes

Signed-off-by: Manan Gupta <[email protected]>

* feat: deprecate disable-replication-manager flag

Signed-off-by: Manan Gupta <[email protected]>

* test: fix start vtorc function to be idempotent

Signed-off-by: Manan Gupta <[email protected]>

* docs: fix review comments

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix port used by vtorc

Signed-off-by: Manan Gupta <[email protected]>

* test: really fix the ports issue

Signed-off-by: Manan Gupta <[email protected]>

* test: fix upgrade downgrade test issues with running vtorc

Signed-off-by: Manan Gupta <[email protected]>

* test: run ISP before starting VTOrc

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix flakines in throttler test due to vtorc

Signed-off-by: Manan Gupta <[email protected]>

* test: fix flakiness in vreplication tests

Signed-off-by: Manan Gupta <[email protected]>

---------

Signed-off-by: Manan Gupta <[email protected]>

* Allow manually kicking off CodeQL (#12200)

* Move vtorc from go-sqlite3 to modernc.org/sqlite (#12214)

* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* Update the MySQL version used by our Docker images (#12054)

* Change default MySQL version of lite and base to mysql80 instead of mysql57

Signed-off-by: Florent Poinsard <[email protected]>

* Bump MySQL version to 8.0.31 in install_dependencies.sh

Signed-off-by: Florent Poinsard <[email protected]>

* addition of label to the mysql80 image

Signed-off-by: Florent Poinsard <[email protected]>

* apply review suggestions

Signed-off-by: Florent Poinsard <[email protected]>

* bring back mysql80 label

Signed-off-by: Florent Poinsard <[email protected]>

* add release summary

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>

* Revert "Use ubuntu 20.04 for release builds (#12202)" (#12220)

The underlying issue has now been addressed in:
  https://github.com/vitessio/vitess/pull/12214

This reverts commit 4b3b37d41ec08130d162b5115904108cbb3dc14b.

Signed-off-by: Matt Lord <[email protected]>

* Update release notes summary for the new default MySQL version (#12222)

Signed-off-by: Florent Poinsard <[email protected]>

* Revert changes made in #11960 (#12219)

Signed-off-by: Florent Poinsard <[email protected]>

* Bug fix: Cache filtered out tablets in topology watcher to avoid unnecessary GetTablet calls to topo (#12194)

* check filter later in loadTablets. Add tests to confirm expected behavior

Signed-off-by: Brian Ramos <[email protected]>

* remove unnecessary assignment

Co-authored-by: Florent Poinsard <[email protected]>
Signed-off-by: Brian Ramos <[email protected]>

* add some explanation around TestFilterByKeypsaceSkipsIgnoredTablets

Signed-off-by: Brian Ramos <[email protected]>

* simplify filter check logic as well as major test cleanup, using testify where possible

Signed-off-by: Brian Ramos <[email protected]>

---------

Signed-off-by: Brian Ramos <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* Update Go dependencies (#12215)

* Update Go dependencies

This updates the Go dependencies to the latest versions before the
Vitess 16 freeze so we can start with an up to date list.

Also updates vitess-mixin since Dependabot is reporting a bunch of
things on that.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix deprecated sets.String

This is now available using generics.

Signed-off-by: Dirkjan Bussink <[email protected]>

* run make proto

Signed-off-by: Florent Poinsard <[email protected]>

* fix e2e test expectations

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* Properly test healthcheck cache response handling. (#12226)

Signed-off-by: Matt Lord <[email protected]>

* vtgate advertised mysql server version to 8.0.31 (#11989)

* change vtgate advertised mysql version to 8.0.31

Signed-off-by: Harshit Gangal <[email protected]>

* refactor keeping mysql advertised version

Signed-off-by: Harshit Gangal <[email protected]>

* refactor sqlparser mysql version to use the mysql_server_version default

Signed-off-by: Harshit Gangal <[email protected]>

* test: update test setup

Signed-off-by: Harshit Gangal <[email protected]>

* test: updated test help output

Signed-off-by: Harshit Gangal <[email protected]>

* added release notes

Signed-off-by: Harshit Gangal <[email protected]>

* feat: remove panic since it is getting triggered from the sidecardb

Signed-off-by: Manan Gupta <[email protected]>

---------

Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Andres Taylor <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>

* schemadiff: fix scenario where no tables exist in schema and with just views reading from DUAL (#12189)

* [Gen4] Fix lookup vindexes with `autocommit` enabled (#12172)

* test: show that `autocommit` on lookup vindexes is broken when used after a savepoint is created

Signed-off-by: Arthur Schreiber <[email protected]>

* test: rework the test case to show that the problem lies with transactions

Signed-off-by: Arthur Schreiber <[email protected]>

* Correctly handle `autocommit` in the `VIndexLookup` primitive.

Signed-off-by: Arthur Schreiber <[email protected]>

* Copy the `logging` struct so autocommit queries show up correctly in `vexplain` output.

Signed-off-by: Arthur Schreiber <[email protected]>

---------

Signed-off-by: Arthur Schreiber <[email protected]>

* Add TOC to the summary docs (#12225)

* docs: add toc to the summary docs

Signed-off-by: Manan Gupta <[email protected]>

* feat: add link to newly added summary change

Signed-off-by: Manan Gupta <[email protected]>

---------

Signed-off-by: Manan Gupta <[email protected]>

* feat: bump reported version of main (#12234)

Signed-off-by: Manan Gupta <[email protected]>

* Use consistent type for vreplication id (#12218)

* Use consistent type for vreplication id

The `_vt.vreplication` table has an `INT` type without `UNSIGNED`, so it
maps to the Go `int32` type to have the exact same consistent range.

This means that we should use `int32` consistently for everywhere that
we use the VReplication identifier.

We had int32, uint32 and int used pretty randomly for these which leads
to CodeQL flagging these and how this was found.

Instead this now converts everything to int32. It's also safe to change
the protobuf, since the protobuf wire format for int32, uint32, int64 &
uint64 is all compatible and actually uses the same variable int length
format.

The only exception would be if someone has a VReplication identifier
that would not fit in `int32`, but well, this PR shows that in that case
we already blow up in unexpected and unknown ways anyway. So it's better
to have that explicitly error.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Add support for all integer types for bind variables

This ensures we cover all the Go int types to generate bind parameters
from.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Regenerate protos for vtadmin

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix test for using ParseInt

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix typecasting issue for workflow types (#12217)

* Fix typecasting issue for workflow types

This addresses a number of the issues identified by CodeQL around
typecasting. We use 32 bit integers for these, so let's make sure we
parse and cast then as those everywhere it matters.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Ensure to write to int into query

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* Use the correct uint32 type for the GTID file position (#12229)

This is defined as uint32 at the protocol level, so better to use that
type consistently and also parse it accordingly with `ParseUint` with
the right size.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix CodeQL identified issues (#12199)

This addresses two classes of issues that CodeQL found. One is a not as
strict as needed regexp for a domain match, where there's no literal `.`
being matched.

The other is a user controlled XSS since we run the template with
arbitrary user input. Instead we validate if the throttler with that
name actually exists before trying to execute the template.

On error we also don't reflect the input to avoid the same XSS.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Copy remaining query serving design docs (#12170)

* Copy remaining query serving design docs

Signed-off-by: Florent Poinsard <[email protected]>

* Remove duplicate design docs

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>

* Use a working MariaDB mirror for CI (#12253)

Signed-off-by: Dirkjan Bussink <[email protected]>

* Update the release instructions after v16 code freeze (#12245)

Signed-off-by: Florent Poinsard <[email protected]>

* onlineddl_vrepl suite: fix auto_increment flakyness (#12246)

* onlineddl_vrepl suite: fix auto_increment flakyness

Signed-off-by: Shlomi Noach <[email protected]>

* delete all rows before events begin

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Revert default MySQL 80 version to `8.0.30` (#12252)

* Revert default MySQL 80 version

Signed-off-by: Florent Poinsard <[email protected]>

* upgrade mysql80 default to 8.0.30

Signed-off-by: Florent Poinsard <[email protected]>

* Add release notes regarding MySQL upgrade path in the vitess-operator

Signed-off-by: Florent Poinsard <[email protected]>

* Remove unwanted upgrade path step

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>

* Fix additional integer type handling (#12237)

* Fix additional integer type handling

The tablet uid can be a uint32 everywhere and we can parse it like that
with pflag as well. This way we don't have to cast and change the valid
range and get correct messages when it's out of range.

As well we use a regular int for the port which better matches how Go
itself tracks ports and removes a bunch of casts from places where we
already using int.

Lastly it also fixes some small cases of how we parse to match the type
we store it in.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix parsing of server id for Zookeeper

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix help output

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* grpcvtgateconn: add Dial and deprecate DialWithOpts (#12210)

Signed-off-by: Matt Layher <[email protected]>
Signed-off-by: Dirkjan Bussink <[email protected]>
Co-authored-by: Dirkjan Bussink <[email protected]>

* Use consistent type port and replication state (#12248)

* Use consistent type port and replication state

These are used as int32 in the protobuf (and enum maps to int32 for
replication state). This means we should also keep using int32
internally if easily possible.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix assertions in test

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* evalengine: new evaluation framework (#12247)

* evalengine: new evaluation framework

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: add cached_size.go

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: simplify casts in NullSafeHashCodeInPlace

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix static warnings

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: remove duplicate branch

Signed-off-by: Vicent Marti <[email protected]>

* planbuilder: update for tuple bind variables

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix error in tuple hashing

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix LIKE simplification

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: cleanup unused tests and panics

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>

* Examples: Add lib functions to wait for shard states (#12239)

* Add lib functions to wait for shard states

Signed-off-by: Matt Lord <[email protected]>

* Fixup region example too

Signed-off-by: Matt Lord <[email protected]>

* Address minor review comments

Signed-off-by: Matt Lord <[email protected]>

* Remove extraneous line change

Signed-off-by: Matt Lord <[email protected]>

* Refactor to remove duplicated files/code

Also address shellcheck warnings.

Signed-off-by: Matt Lord <[email protected]>

* Fix tests that run in CI

Signed-off-by: Matt Lord <[email protected]>

* Use more robust location detection

Signed-off-by: Matt Lord <[email protected]>

* Fix backup tests

Signed-off-by: Matt Lord <[email protected]>

* Exit on shard healthy failure

Signed-off-by: Matt Lord <[email protected]>

* Remove unused dups and adjust client_test and Dockerfiles

Signed-off-by: Matt Lord <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>

* Download mariadb from vitess-resources (#12271)

* Download mariadb from vitess-resources

Signed-off-by: Florent Poinsard <[email protected]>

* Use proper mariadb version increment in tests

Signed-off-by: Florent Poinsard <[email protected]>

* Use 10.10.3 instead of LTS

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>

* Workflow Show: display rows copied  (#12231)

* Display rows copied in Workflow Show

Signed-off-by: Rohit Nayak <[email protected]>

* Fix vtctl tests

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Rohit Nayak <[email protected]>

* sqlparser: Add parsing support to spatial POINT expression (#12198)

* feat: add parsing support to spatial POINT expression

Signed-off-by: Ayman161803 <[email protected]>

* fix: address formatting issues

Signed-off-by: Ayman161803 <[email protected]>

* fix: address formatting issues

Signed-off-by: Ayman <[email protected]>

* feat: add canonical test cases for spatial point expression

Signed-off-by: Ayman161803 <[email protected]>

---------

Signed-off-by: Ayman161803 <[email protected]>
Signed-off-by: Ayman <[email protected]>

* fix dual table handling (#12204)

* fix planner: be careful when merging subqueries

We were not checking dependencies of the outer subquery correctly,
before merging with a route. This lead to invalid plans being produced.

Signed-off-by: Andres Taylor <[email protected]>

* fix planner: use tables on lhs when merging subqs

When calculating if everything we need is available before
merging routes, we now take into account anything coming
from the LHS of any joins we are under.

Signed-off-by: Andres Taylor <[email protected]>

* test: add test merging subquery with join

Signed-off-by: Andres Taylor <[email protected]>

* use LHS from the top of the current node when planning RHSs

Signed-off-by: Florent Poinsard <[email protected]>

* added comments

Signed-off-by: Andres Taylor <[email protected]>

---------

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* Subquery bugfix (#12254)

* fix planner: be careful when merging subqueries

We were not checking dependencies of the outer subquery correctly,
before merging with a route. This lead to invalid plans being produced.

Signed-off-by: Andres Taylor <[email protected]>

* fix planner: use tables on lhs when merging subqs

When calculating if everything we need is available before
merging routes, we now take into account anything coming
from the LHS of any joins we are under.

Signed-off-by: Andres Taylor <[email protected]>

* test: add test merging subquery with join

Signed-off-by: Andres Taylor <[email protected]>

* use LHS from the top of the current node when planning RHSs

Signed-off-by: Florent Poinsard <[email protected]>

* added comments

Signed-off-by: Andres Taylor <[email protected]>

---------

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* feat: fix release script (#12284)

Signed-off-by: Manan Gupta <[email protected]>

* feat: copy release notes from release-16 (#12287)

Signed-off-by: Manan Gupta <[email protected]>

* Remove parallelization from tests that mutate shared state (#12257)

Signed-off-by: Andrew Mason <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* fix dual table handling (#12292)

* fix dual table handling

Signed-off-by: Andres Taylor <[email protected]>

* update test expectations

Signed-off-by: Andres Taylor <[email protected]>

---------

Signed-off-by: Andres Taylor <[email protected]>

* fix: transaction_isolation to be applied at session level (#12281)

Signed-off-by: Harshit Gangal <[email protected]>

* fix linter warnings (#12286)

* fix linter warnings

Signed-off-by: Andres Taylor <[email protected]>

* add the same checks in CI as pre-commit

Signed-off-by: Andres Taylor <[email protected]>

* ci: disable revive by default

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Vicent Marti <[email protected]>
Co-authored-by: Vicent Marti <[email protected]>

* go/vt/mysqlctl: add configurable read buffer to builtin backups (#12073)

* go/vt/mysqlctl: add configurable read buffer to builtin backups

Signed-off-by: Max Englander <[email protected]>

* pr review: use uint

Signed-off-by: Max Englander <[email protected]>

* doc/releasenotes: v16 --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Signed-off-by: Max Englander <[email protected]>

* update flags txt

Signed-off-by: Max Englander <[email protected]>

* make part of v17

Signed-off-by: Max Englander <[email protected]>

* Update doc/releasenotes/17_0_0_summary.md

Co-authored-by: Deepthi Sigireddi <[email protected]>
Signed-off-by: Max Englander <[email protected]>

* improve wording -block

Signed-off-by: Max Englander <[email protected]>

---------

Signed-off-by: Max Englander <[email protected]>
Signed-off-by: Max Englander <[email protected]>
Co-authored-by: Deepthi Sigireddi <[email protected]>

* VReplication Last Error: retry error if it happens after timeout (#12114)

* Retry error if it happens after timeout

Signed-off-by: Rohit Nayak <[email protected]>

* Address review comments

Signed-off-by: Rohit Nayak <[email protected]>

* Address review comments

Signed-off-by: Rohit Nayak <[email protected]>

* Improve unit tests

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Rohit Nayak <[email protected]>

* fix: added null safe operator precendence rule (#12297)

Signed-off-by: Harshit Gangal <[email protected]>

* Fix aggregation on outer joins (#12298)

* fix: aggregation coming from right hand side of left join check for null

Signed-off-by: Harshit Gangal <[email protected]>

* test: added a sum test

Signed-off-by: Harshit Gangal <[email protected]>

* addressed review comments

Signed-off-by: Harshit Gangal <[email protected]>

---------

Signed-off-by: Harshit Gangal <[email protected]>

* Change SQL error codes to use explicit type (#12249)

This ErrorCode type maps to a uint16 to match what MySQL has as the
range and parses things accordingly as well. With the explicit type we
know better when we use the specific error code.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix typecasting alerts inside vtgate engine (#12238)

CodeQL reported a few cases in the vtgate engine where we can simplify
and make the typecasting more correct.

We were double casting for the eval as well which isn't useful, so
reduce this to it doing one for the hash code.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Move to independent sets package (#12251)

This moves to a separate sets package inside Vitess so we don't break
version pinning for k8s like in the Vitess operator.

It mostly copies the existing implementation, but simplifies and cleans
it up to only copy what we actually use.

Signed-off-by: Dirkjan Bussink <[email protected]>

* SchemaDiff: normalize boolean columns to tinyint(1), and, add TableQualifier hint to force adding a db qualifier to the alter diff. (#12133)

* Normalize boolean columns to tinyint(1)

Signed-off-by: Rohit Nayak <[email protected]>

* Properly handle invalid defaults provided for boolean columns

Signed-off-by: Rohit Nayak <[email protected]>

* Use TableQualifier hint to decide whether to use the db qualifier for the tables in create/alter scripts

Signed-off-by: Rohit Nayak <[email protected]>

* Keep current behavior as default and require TableQualifierDeclared to force use of the 'to' desired schema's qualifier

Signed-off-by: Rohit Nayak <[email protected]>

* Fix incorrect renaming of enum in comments

Signed-off-by: Rohit Nayak <[email protected]>

* Additional tests for boolean/tinyint(1) equivalence

Signed-off-by: Rohit Nayak <[email protected]>

* Add tests for create/drop with table qualifiers

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Rohit Nayak <[email protected]>

* Allow zero (in) date when setting up internal _vt schema (#12262)

* Allow zero (in) date when setting up internal _vt schema

Signed-off-by: Shlomi Noach <[email protected]>

* modify test sto include @@sql_mod query support

Signed-off-by: Shlomi Noach <[email protected]>

* Allow zero (in) date when setting up internal _vt schema

Signed-off-by: Shlomi Noach <[email protected]>

* modify test sto include @@sql_mod query support

Signed-off-by: Shlomi Noach <[email protected]>

* Fix test failures

Signed-off-by: Rohit Nayak <[email protected]>

* Fix failing tests. Minor refactor

Signed-off-by: Rohit Nayak <[email protected]>

* change sql_mode t omost permissive (empty)

Signed-off-by: Shlomi Noach <[email protected]>

* setPermissiveSQLMode

Signed-off-by: Shlomi Noach <[email protected]>

* fixes per review

Signed-off-by: Shlomi Noach <[email protected]>

* Add missing mock query

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>
Signed-off-by: Rohit Nayak <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>

* maintainers: add dbussink and rework some sections (#12329)

Signed-off-by: Vicent Marti <[email protected]>

* TableFilter: fix excluded RE listing (#12318)

Signed-off-by: Shlomi Noach <[email protected]>

* Internal refactor: LastError as a public struct (#12321)

* Refactor LastError as a public struct

Signed-off-by: Shlomi Noach <[email protected]>

* wording

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* evalengine: it's time for JSON! (#12274)

* collations: add max width to charsets

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: add support for JSON built-ins

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support JSON_ARRAY and JSON_OBJECT

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: JSON object sorting like MySQL

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: add support for CAST(val AS JSON)

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support TO_BASE64 and FROM_BASE64

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support JSON_DEPTH

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support JSON_LENGTH

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix codegen

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: do not wrap `json.Value`

Signed-off-by: Vicent Marti <[email protected]>

* sizegen: fix external type generation

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support JSON_CONTAINS_PATH

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: support JSON_KEYS

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix linter warnings

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix evalToJSON semantics

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix error wrapping

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: update sizegen

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: @systay review feedback

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: oops! fix bad else/if

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix bugs with decimal rounding (#12360)

Signed-off-by: Vicent Marti <[email protected]>

* VReplication: ignore GC tables in schema analysis (#12320)

* TableFilter: fix excluded RE listing

Signed-off-by: Shlomi Noach <[email protected]>

* extract GCTableNameExpression

Signed-off-by: Shlomi Noach <[email protected]>

* VReplication: exclude GC tables when reading schema

Signed-off-by: Shlomi Noach <[email protected]>

* wrap schema analyzis errors for better tracability

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Add parsing support for linestring constructor (#12299)

* feat: add parsing support to linestring constructor

Signed-off-by: Ayman161803 <[email protected]>

* test: add canonical tests

Signed-off-by: Ayman161803 <[email protected]>

* fix: fix formatting issues

Signed-off-by: Ayman161803 <[email protected]>

* chore: increment shift/reduce conflicts to 4

Signed-off-by: Ayman161803 <[email protected]>

* fix: fix shift/reduce conflict by adding precendence

Signed-off-by: Ayman161803 <[email protected]>

* fix: update yacc grammar to replace []pointExpr with Exprs

Signed-off-by: Ayman161803 <[email protected]>

* fix: update format function for linestringExpr and select_test_cases

Signed-off-by: Ayman161803 <[email protected]>

---------

Signed-off-by: Ayman161803 <[email protected]>

* testutils: use a rows parser for test comparisons (#12138)

* tesutils: use a rows parser for test comparisons

Signed-off-by: Vicent Marti <[email protected]>

* utils: add license

Signed-off-by: Vicent Marti <[email protected]>

* moved parse_rows to sqltypes to avoid import cycle

Signed-off-by: Andres Taylor <[email protected]>

* concatenate: fix test

Signed-off-by: Vicent Marti <[email protected]>

* sqltypes: fix comparison error with duplicate rows

Signed-off-by: Vicent Marti <[email protected]>

* vexplain: fix undeterministic test comparisons

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Co-authored-by: Andres Taylor <[email protected]>

* vtbackup, mysqlctl: detailed backup and restore metrics (#11979)

* vtbackup, mysqlctl: detailed backup and restore metrics

While backup stats are exposed via `servenv` (which is great!) ideally we would
like more fine-grained stats on:

 * Download last backup.
 * Decompress last backup.
 * Catchup to primary with replication.
 * Compress new backup.
 * Upload new backup.

Signed-off-by: Max Englander <[email protected]>

* move fakemysqldaemon.go to mysqlctl package

Signed-off-by: Max Englander <[email protected]>

* go/vt/mysqlctl: add backup/restore stats tests

Signed-off-by: Max Englander <[email protected]>

* address pr comments round 1

Signed-off-by: Max Englander <[email protected]>

* checkpoint

Signed-off-by: Max Englander <[email protected]>

* reorganize metric names

Signed-off-by: Max Englander <[email protected]>

* track bytes processed in backup/restore

Signed-off-by: Max Englander <[email protected]>

* address pr comments: remove 'this file' from comments

Signed-off-by: Max Englander <[email protected]>

* Update go/vt/mysqlctl/backupstats/stats.go

Co-authored-by: Deepthi Sigireddi <[email protected]>
Signed-off-by: Max Englander <[email protected]>

* Update go/vt/mysqlctl/backupstats/stats.go

Co-authored-by: Deepthi Sigireddi <[email protected]>
Signed-off-by: Max Englander <[email protected]>

* Update go/vt/mysqlctl/backupstats/stats.go

Co-authored-by: Deepthi Sigireddi <[email protected]>
Signed-off-by: Max Englander <[email protected]>

* pr review: improve comments

Signed-off-by: Max Englander <[email protected]>

* make stats more expvars-friendly

Signed-off-by: Max Englander <[email protected]>

* -backupstorage.Parameterizer, +backupstorage.BackupStorage.WithParams

Signed-off-by: Max Englander <[email protected]>

* remove deleted field

Signed-off-by: Max Englander <[email protected]>

* Update go/ioutil/meter.go

Signed-off-by: Max Englander <[email protected]>

* Update go/ioutil/meter.go

Signed-off-by: Max Englander <[email protected]>

* Update go/ioutil/meter.go

Signed-off-by: Max Englander <[email protected]>

* -constants.go +release notes

Signed-off-by: Max Englander <[email protected]>

* add stats example to release notes

Signed-off-by: Max Englander <[email protected]>

---------

Signed-off-by: Max Englander <[email protected]>
Signed-off-by: Max Englander <[email protected]>
Co-authored-by: Deepthi Sigireddi <[email protected]>

* VReplication: workflows with multiple streams fail to start on multi-primary setups (#12228)

* Handle auto_increment_increment step > 1 while creating multiple vreplication streams

Signed-off-by: Rohit Nayak <[email protected]>

* Enhance e2e test for validating auto_increment step logic

Signed-off-by: Rohit Nayak <[email protected]>

* Add new query to mock db

Signed-off-by: Rohit Nayak <[email protected]>

* Add missing mock for TestEngineExec

Signed-off-by: Rohit Nayak <[email protected]>

* Cleanup

Signed-off-by: Rohit Nayak <[email protected]>

* Fix goimports

Signed-off-by: Rohit Nayak <[email protected]>

* Address review comments

Signed-off-by: Rohit Nayak <[email protected]>

* Use uint16 for auto increment step

Signed-off-by: Rohit Nayak <[email protected]>

* Update comments

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Rohit Nayak <[email protected]>

* sqlparser/schemadiff: normalize CAST type case (#12372)

Signed-off-by: Shlomi Noach <[email protected]>

* add database name to _vt.views table (#12374)

Signed-off-by: Harshit Gangal <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>

* [vtgate planner] Routing & Merging refactor (#12197)

* start of extracting routing logic out from Route into an interface

Signed-off-by: Andres Taylor <[email protected]>

* wip - milestone - first plan-test green

Signed-off-by: Andres Taylor <[email protected]>

* wip - single table routes work fine

Signed-off-by: Andres Taylor <[email protected]>

* wip - sequence tables passing

Signed-off-by: Andres Taylor <[email protected]>

* wip - all tests in from_cases.json passing

Signed-off-by: Andres Taylor <[email protected]>

* wip - more work on merge logic

Signed-off-by: Andres Taylor <[email protected]>

* wip - need to switch the routes as well

Signed-off-by: Andres Taylor <[email protected]>

* subquery merging

Signed-off-by: Harshit Gangal <[email protected]>

* dual routing

Signed-off-by: Harshit Gangal <[email protected]>

* wip - make sure to keep inner/outer in the right place

Signed-off-by: Andres Taylor <[email protected]>

* make sure to not forget already seen predicates when merging sharded routes

Signed-off-by: Andres Taylor <[email protected]>

* add merged subqueries to the merged field

Signed-off-by: Andres Taylor <[email protected]>

* dual subquery and none routing change

Signed-off-by: Harshit Gangal <[email protected]>

* handle merging subqueries on IN comparisons

Signed-off-by: Andres Taylor <[email protected]>

* recalculate routing after merging subquery

Signed-off-by: Andres Taylor <[email protected]>

* rename field to make it easier to grokk

Signed-off-by: Andres Taylor <[email protected]>

* better merging logic when merging non-sharded tables

Signed-off-by: Andres Taylor <[email protected]>

* when merging subqueries, we must also copy predicates

Signed-off-by: Andres Taylor <[email protected]>

* copy keyspace when producing NoneRouting

Signed-off-by: Andres Taylor <[email protected]>

* implement Routing methods in TargetedRouting and update OpCode outside of shardedrouting

Signed-off-by: Florent Poinsard <[email protected]>

* Fix targeted routing update params and rest of the dml cases

Signed-off-by: Florent Poinsard <[email protected]>

* Enhanced UpdateRoutingLogic function to detect constant null and None routes

Signed-off-by: Florent Poinsard <[email protected]>

* clean up info schema route mergeing

Signed-off-by: Andres Taylor <[email protected]>

* handle reference tables with alternates

Signed-off-by: Andres Taylor <[email protected]>

* update remaining plan_tests

Signed-off-by: Andres Taylor <[email protected]>

* unify unsharded and reference routing in the same logic

Signed-off-by: Andres Taylor <[email protected]>

* clean up merging.go

Signed-off-by: Andres Taylor <[email protected]>

* build the alternate routes with the correct keyspace

Signed-off-by: Andres Taylor <[email protected]>

* minor fixes after self-review

Signed-off-by: Andres Taylor <[email protected]>

* when missing current keyspace, any valid keyspace can be used

Signed-off-by: Andres Taylor <[email protected]>

* final small changes. I promise

Signed-off-by: Andres Taylor <[email protected]>

* review feedback

Signed-off-by: Andres Taylor <[email protected]>

* tidy up method after review feedback

Signed-off-by: Andres Taylor <[email protected]>

* more cleanup - fix goland warnings in new files

Signed-off-by: Andres Taylor <[email protected]>

* add keyspace information to sequence routing

Signed-off-by: Andres Taylor <[email protected]>

* more cleanups from PR review

Signed-off-by: Andres Taylor <[email protected]>

* more cleanup

Signed-off-by: Andres Taylor <[email protected]>

---------

Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* This PR bumps the vitess resrouces release increment from v2 to v4 (#12272)

Signed-off-by: Florent Poinsard <[email protected]>

* Switch to using new Go 1.19 CRL parser (#12315)

`x509.ParseCRL` is deprecated, we should use `x509.ParseRevocationList`
instead which is new in Go 1.19.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Online DDL: remove legacy "stowaway table" logic (#12288)

* Online DDL: remove legacy stowaway table logic

Signed-off-by: Shlomi Noach <[email protected]>

* removed column from table definition

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Fix for USING when column names not lower cased (#12378)

Signed-off-by: Andres Taylor <[email protected]>

* Fix bug in vtexplain around JOINs (#12376)

Signed-off-by: Andres Taylor <[email protected]>

* fix help text for backup_storage_number_blocks (#12258)

Signed-off-by: Rameez Sajwani <[email protected]>

* Fix additional typecasting alerts (#12309)

* Fix additional typecasting alerts

This should solve the remaining valid typecasting alerts from CodeQL.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix additional typecasting issues

Ran through more of the ParseInt / ParseUint / Atoi calls manually that
CodeQL didn't identify and found a bunch more inconsistencies that
should be fixed.

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix additional casting bugs in the sqlparser

This fixes a bunch of additional canonical casing bugs in the parser.
See https://github.com/vitessio/vitess/issues/12392 for the list of
issues it addresses.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fixes for local docker environment (#12382)

Signed-off-by: Andy Lim <[email protected]>

* [main] Schema RPC to fetch table/view definition (#12375) (#12397)

* feat: proto changes to add new rpc

Signed-off-by: Harshit Gangal <[email protected]>

* feat: vttablet changes in queryservice to implement the new rpc

Signed-off-by: Harshit Gangal <[email protected]>

* added test on tabletserver

Signed-off-by: Harshit Gangal <[email protected]>

* feat: use the new schema rpc in vtgate to get updated view definition

Signed-off-by: Harshit Gangal <[email protected]>

* remove error from getschemaresponse and change the enum name

Signed-off-by: Harshit Gangal <[email protected]>

* updated proto files for vtadmin

Signed-off-by: Harshit Gangal <[email protected]>

* fix e2e test

Signed-off-by: Harshit Gangal <[email protected]>

---------

Signed-off-by: Harshit Gangal <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>

* Bump golang.org/x/net from 0.5.0 to 0.7.0 (#12390)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Move to Go 1.19 atomics (#12391)

* Move to Go 1.19 atomics

This change moves to Go 1.19 atomics in favor of the version we had in
the `sync2` package. The version in that package predates the
availability of the new atomics, but for the future we want to use the
Go provided ones.

On top of that, we can also use the semaphore from `x/sync` instead of a
custom one in `sync2`. This cleans up a bunch of additional code from
`sync2`.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Don't assume the value is a duration

This can be any float, so we need to store the float bits in an atomic
uint64 instead (as Go has no atomic floats).

Signed-off-by: Dirkjan Bussink <[email protected]>

* Log actual error if acquiring semaphore fails

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix overlooked metrics threshold case

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* Update additional Go dependencies (#12401)

Dependabot does pick up these go.mod files but somehow fails to open PRs
for them to update things that are vulnerable. So doing it manually
here.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix another mixin dep (#12403)

Dependabot is still warning about this one and missed this in #12401

Signed-off-by: Dirkjan Bussink <[email protected]>

* Remove a bunch of debug logging (#12404)

This removes a bunch of debug logging that is set for higher log levels.
The problem is that many of these are logging user provided values
directly which is something CodeQL also alerts on.

If someone needs this kind of additional logging level, I think it
suffices to modify the code locally to test something rather than always
carrying these high log level entries.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Stop launchable integration for unit tests (#12386)

* feat: stop launchable integration for unit tests

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix identation for the unit tests

Signed-off-by: Manan Gupta <[email protected]>

* feat: also add name to the unit tests

Signed-off-by: Manan Gupta <[email protected]>

---------

Signed-off-by: Manan Gupta <[email protected]>

* Makefile: use buildx when GOOS/GOARCH are specified (#12081)

Signed-off-by: Max Englander <[email protected]>

* Auto refresh tables on vtgate Status Page (#12335)

* Component: vtgate, Auto Update tables in Status For Vtgate page

Signed-off-by: jun <[email protected]>

* rename function name

Signed-off-by: jun <[email protected]>

* move tableRefreshRate to servenv.RegisterFlags()

Signed-off-by: jun <[email protected]>

* Replace JQuery with Vanilla javascript && set default value 0 as non-refreshing && update e2e .txt flags

Signed-off-by: jun <[email protected]>

* update go/flags/endtoend/*.txt

Signed-off-by: jun <[email protected]>

* update e2e flags txt files

Signed-off-by: jun <[email protected]>

* update --table-refresh-interval tag with proper position

Signed-off-by: jun <[email protected]>

* remove extra space in mysqlctl.txt vtorc.txt
repush

Signed-off-by: jun <[email protected]>

---------

Signed-off-by: jun <[email protected]>
Co-authored-by: jun <[email protected]>

* (re)Formalize SQLError in VReplication, add underlying wrap/unwrap functionality (#12327)

* (re)Formalize SQLError in VReplication, add underlying wrap/unwrap functionality

Signed-off-by: Shlomi Noach <[email protected]>

* update test to match new UnwrapAll() behavior

Signed-off-by: Shlomi Noach <[email protected]>

* do not unwrap

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Online DDL: improve retry of vreplication errors with `vitess` `ALTER TABLE` migrations (#12323)

* Refactor LastError as a public struct

Signed-off-by: Shlomi Noach <[email protected]>

* Online DDL: use LastError for retrying vreplication streams

Signed-off-by: Shlomi Noach <[email protected]>

* differentiate between a temrinally failed workflow and one which 'has error' and can be retried

Signed-off-by: Shlomi Noach <[email protected]>

* wording

Signed-off-by: Shlomi Noach <[email protected]>

* simplified hasError()

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Fixing `onlineddl_vrepl` flakiness, and adding more tests (#12325)

* TableFilter: fix excluded RE listing

Signed-off-by: Shlomi Noach <[email protected]>

* extract GCTableNameExpression

Signed-off-by: Shlomi Noach <[email protected]>

* VReplication: exclude GC tables when reading schema

Signed-off-by: Shlomi Noach <[email protected]>

* wrap schema analyzis errors for better tracability

Signed-off-by: Shlomi Noach <[email protected]>

* Refactor LastError as a public struct

Signed-off-by: Shlomi Noach <[email protected]>

* Online DDL: use LastError for retrying vreplication streams

Signed-off-by: Shlomi Noach <[email protected]>

* refactor UpdateThrottlerTopoConfig

Signed-off-by: Shlomi Noach <[email protected]>

* onlineddl_vrepl tests:
- using topo-based throttler config
- break down of throttle-based PRS test
- adding --postpone-completion -based PRS test
- fixing wait for VReplicatoin status

Signed-off-by: Shlomi Noach <[email protected]>

* differentiate between a temrinally failed workflow and one which 'has error' and can be retried

Signed-off-by: Shlomi Noach <[email protected]>

* (re)Formalize SQLError in VReplication, add underlying wrap/unwrap functionality

Signed-off-by: Shlomi Noach <[email protected]>

* differentiate between a temrinally failed workflow and one which 'has error' and can be retried

Signed-off-by: Shlomi Noach <[email protected]>

* wording

Signed-off-by: Shlomi Noach <[email protected]>

* simplified hasError()

Signed-off-by: Shlomi Noach <[email protected]>

* do not unwrap

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Cleanup vtorc ssl package (#12423)

This is not used for anything anymore, so let's remove it.

Signed-off-by: Dirkjan Bussink <[email protected]>

* SidecarDB Init: don't fail on schema init errors (#12328)

* Don't fail on schema init errors

Signed-off-by: Rohit Nayak <[email protected]>

* Fix debug var attribute name

Signed-off-by: Rohit Nayak <[email protected]>

* Fix typo in debug var

Signed-off-by: Rohit Nayak <[email protected]>

* Address review comment

Signed-off-by: Rohit Nayak <[email protected]>

* Address review comments

Signed-off-by: Rohit Nayak <[email protected]>

---------

Signed-off-by: Rohit Nayak <[email protected]>

* [main] Examples: Correct VTAdmin Discovery File Path And Add Check (#12415) (#12430)

* Examples: correct VTAdmin discovery file path and add check

Signed-off-by: Matt Lord <[email protected]>

* Remove jq dependency

Signed-off-by: Matt Lord <[email protected]>

* vtadmin examples scripts hardcode localhost

Signed-off-by: Matt Lord <[email protected]>

* Fix errant expectation

Signed-off-by: Matt Lord <[email protected]>

* Adjust healthy shard indicator

Signed-off-by: Matt Lord <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>
Co-authored-by: Matt Lord <[email protected]>

* sqlparser: Tools cleanup (#12407)

* tools: fix compilation checks in ast helpers

Signed-off-by: Vicent Marti <[email protected]>

* sqlparser: simplify generation

Signed-off-by: Vicent Marti <[email protected]>

* tools: make sizegen more resilient

Signed-off-by: Vicent Marti <[email protected]>

* tools: fix formatting of all generated code

Signed-off-by: Vicent Marti <[email protected]>

* tools: fix call

Signed-off-by: Vicent Marti <[email protected]>

* tools: review feedback

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>

* Add configuration option for minimum TLS version (#12424)

* Add configuration option for minimum TLS version

We were hardcoding this to TLS 1.0 which is not considered safe anymore
these days and was called out by CodeQL as a security issue.

Instead we should default to TLS 1.2 which we do everywhere else, and
allow for an option to change it if needed so that we're safe by default
here.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Add release notes item

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* VReplication: Use MariaDB Compat JSON Functions (#12434)

This is needed to better support importing from
MariaDB into Vitess/MySQL.

Signed-off-by: Matt Lord <[email protected]>
Co-authored-by: Matt Lord <[email protected]>

* Remove are-you-alive example (#12432)

Signed-off-by: Florent Poinsard <[email protected]>

* schemadiff: AlterTableAlgorithmStrategy (#12442)

* schemadiff: AlterTableAlgorithmStratgy

Signed-off-by: Shlomi Noach <[email protected]>

* minor rename

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* Add release notes summary for views (#12422) (#12425)

* Add release notes summary for support views



* add link to the documentation



* simplify views release notes



---------

Signed-off-by: Florent Poinsard <[email protected]>

* [main] Change `GetSchema` RPC to return `CreateView` instead of `SelectStmt` (#12421) (#12441)

* Changed getSchemaRPC to return CreateView instead of SelectStmt

Signed-off-by: Florent Poinsard <[email protected]>

* test: fix expectation

Signed-off-by: Harshit Gangal <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Harshit Gangal <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>

* Add 128 bit hashing for the evalengine (#12428)

* evalengine: 128-bit hashes

This adds support for 128 bit hashes. With 128 bit hashes, we have
enough entropy so we can remove the need to compare values after
hashing.

Signed-off-by: Vicent Marti <[email protected]>
Signed-off-by: Dirkjan Bussink <[email protected]>

* evalengine: Add support for 128 bit hashing

This adds a null safe hashing function for 128 bit hashing and tests it.
There's no usage of this yet, but that will be extended in the future.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Address review feedback

Signed-off-by: Dirkjan Bussink <[email protected]>

* Ensure to always initialize hasher

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>
Signed-off-by: Dirkjan Bussink <[email protected]>
Co-authored-by: Vicent Martí <[email protected]>

* feat: update the pull request template (#12443)

Signed-off-by: Manan Gupta <[email protected]>

* evalengine: use 128 bit hashing internally (#12452)

* evalengine: use 128 bit hashing internally

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: implement hashable for hashable types

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: update sizegen

Signed-off-by: Vicent Marti <[email protected]>

* evalengine: fix error with unsupported collations

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>

* Fix Vtop example and release script (#12457)

* examples: should use mysql80 property for vitess image

Signed-off-by: deepthi <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>

* Use MySQL80 in all examples/operator YAML files

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>

* feat: update release script to update vtadmin image version too

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix vtop example script to use vtctldclient instead

Signed-off-by: Manan Gupta <[email protected]>

* feat: update operator.yaml to the latest version

Signed-off-by: Manan Gupta <[email protected]>

* feat: fix yaml files to not pass deprecated parameters

Signed-off-by: Manan Gupta <[email protected]>

* feat: revert changes made for testing

Signed-off-by: Manan Gupta <[email protected]>

---------

Signed-off-by: deepthi <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: deepthi <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* feat: add steps on how to merge during code-freeze (#12444)

Signed-off-by: Manan Gupta <[email protected]>

* [main] GetSchema rpc to streaming api (#12447) (#12459)

* GetSchema rpc to streaming api

Signed-off-by: Harshit Gangal <[email protected]>

* addressed review comment

Signed-off-by: Harshit Gangal <[email protected]>

---------

Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>

* feat: don't keep data for upgrade-downgrade tests (#12465)

Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>

* Create Views allowed for same keyspace (#12409) (#12454)

* restrict views to single keyspace



* feat: also create/alter and drop views on MySQL for information-schema-support



* feat: do the DMLs for alter and create of view in a transaction so that they can be rolled back in case of MySQL failure



* ignore view changes by schema tracker in vttablet when views are enabled



* test: fix test expectation



* Correct syntax error in defer func of E2E test



---------

Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* Add port to enable vtadmin access. (#12467)

Signed-off-by: Andy Lim <[email protected]>

* Fix scalar aggregation engine primitive for column truncation (#12468) (#12473)

* fix: scalar aggregation truncation



* test: added scalar aggr engine unit test



* remove onecase change



---------

Signed-off-by: Harshit Gangal <[email protected]>
Co-authored-by: Harshit Gangal <[email protected]>

* Fix `SHOW VSCHEMA TABLES` tests using v17 vtgate that expected `dual` (#12381) (#12466)

* Fix SHOW VSCHEMA TABLES tests using v17 vtgate



* Fix tests expectations



---------

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>

* examples schema: demi-idempotency via CREATE TABLE IF NOT EXISTS (#12453)

* examples schema: demi-idempotency via CREATE TABLE IF NOT EXISTS

Signed-off-by: Shlomi Noach <[email protected]>

* idempotency: if not exists for all tables under examples/

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* evalengine: add more coercion paths to Hash128 (#12463)

Signed-off-by: Vicent Marti <[email protected]>

* [misc] Delete `automation*` protos (#12449)

* Remove unused automation,automationservice protos and codegen

Signed-off-by: Andrew Mason <[email protected]>

* Release notes

Signed-off-by: Andrew Mason <[email protected]>

---------

Signed-off-by: Andrew Mason <[email protected]>

* BugFix: Unsharded query using a derived table and a dual table (#12484) (#12491)

* test: add failing test for a query in an unsharded keyspace using a derived table and reference table



* feat: fix logic to check if a single unsharded keyspace is used



* feat: fix test output



---------

Signed-off-by: Manan Gupta <[email protected]>

* Make upgrade downgrade job names unique (#12485)

Signed-off-by: Florent Poinsard <[email protected]>

* Add nvm and node to vtadmin-up.sh (#12439)

* Add nvm and node

Signed-off-by: notfelineit <[email protected]>

* Update script with PR recommendations

Signed-off-by: notfelineit <[email protected]>

* Push up local changes

Signed-off-by: notfelineit <[email protected]>

* Update examples/common/scripts/vtadmin-up.sh

Signed-off-by: Frances Thai <[email protected]>

* Update examples/common/scripts/vtadmin-up.sh

Signed-off-by: Frances Thai <[email protected]>

---------

Signed-off-by: notfelineit <[email protected]>
Signed-off-by: Frances Thai <[email protected]>

* Format vtctldclient GetKeyspace output using cli.MarshalJSON. (#12495)

Signed-off-by: Matt Lord <[email protected]>

* ApplySchema: deprecate '--skip_preflight' flag (#10716)

* ApplySchema: deprecate '-skip_preflight' flag

Signed-off-by: Shlomi Noach <[email protected]>

* warn if --skip_preflight is provided

Signed-off-by: Shlomi Noach <[email protected]>

* remove -skip_preflight from tests

Signed-off-by: Shlomi Noach <[email protected]>

* MarkDeprecated, fix usage

Signed-off-by: Shlomi Noach <[email protected]>

* remove skip_preflight test

Signed-off-by: Shlomi Noach <[email protected]>

---------

Signed-off-by: Shlomi Noach <[email protected]>

* streamlog: make generic (#12494)

* streamlog: make generic

Signed-off-by: Vicent Marti <[email protected]>

* gofmt

Signed-off-by: Vicent Marti <[email protected]>

---------

Signed-off-by: Vicent Marti <[email protected]>

* feat: copy release notes from v16 (#12516)

Signed-off-by: Manan Gupta <[email protected]>

* Upgrade to `go1.20.1` (#12399)

* Upgrade to go1.20.1

Signed-off-by: Florent Poinsard <[email protected]>

* Bump golangci-lint version

Signed-off-by: Florent Poinsard <[email protected]>

* Apply linter suggestions

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: deepthi <[email protected]>

* fix TestTLSClientVerifyIdentity

Signed-off-by: Florent Poinsard <[email protected]>

* Revert unwanted change in server_test

Signed-off-by: Florent Poinsard <[email protected]>

* handle readtablet error

Signed-off-by: Florent Poinsard <[email protected]>

* skip TestFuzz

Signed-off-by: Florent Poinsard <[email protected]>

* Unskip TestFuzz and fix issue

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: deepthi <[email protected]>
Co-authored-by: deepthi <[email protected]>

* Fix `vtctldclient`'s Root command to return an error on unknown command (#12481)

* Add a Run func to `vtctldclient`'s Root command to return an error on unknown command

Closes #12480.

Signed-off-by: Andrew Mason <[email protected]>

* Add test

Signed-off-by: Andrew Mason <[email protected]>

* flags test data

Signed-off-by: Andrew Mason <[email protected]>

---------

Signed-off-by: Andrew Mason <[email protected]>

* Flakes: Use new healthy shard check in vreplication e2e tests (#12502)

* Use new healthy shard check in vreplication e2e tests

This is needed because checking that there's a primary tablet for
the shard in vtgate's healtcheck is no longer a reliable indicator
that the shard has a healthy serving primary, because now a
primary needs to initialize its sidecar database and wait for
that to replicate via semi-sync before it becomes serving and can
proceed to perform normal functions. So this delay could cause
test flakiness if you required a healthy shard before continuing
with the test.

Signed-off-by: Matt Lord <[email protected]>

* Try to address unit test race flakes around log size

They looked like this:
WARNING: DATA RACE
Write at 0x000005bf9b60 by goroutine 27141:
  github.com/spf13/pflag.newUint64Value()
      /home/runner/go/pkg/mod/github.com/spf13/[email protected]/uint64.go:9 +0x5a
  github.com/spf13/pflag.(*FlagSet).Uint64Var()
      /home/runner/go/pkg/mod/github.com/spf13/[email protected]/uint64.go:45 +0x55
  vitess.io/vitess/go/vt/log.RegisterFlags()
      /hom…
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request May 29, 2024
* Update Go dependencies

This updates the Go dependencies to the latest versions before the
Vitess 16 freeze so we can start with an up to date list.

Also updates vitess-mixin since Dependabot is reporting a bunch of
things on that.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix deprecated sets.String

This is now available using generics.

Signed-off-by: Dirkjan Bussink <[email protected]>

* run make proto

Signed-off-by: Florent Poinsard <[email protected]>

* fix e2e test expectations

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request May 29, 2024
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request May 29, 2024
* Update Go dependencies

This updates the Go dependencies to the latest versions before the
Vitess 16 freeze so we can start with an up to date list.

Also updates vitess-mixin since Dependabot is reporting a bunch of
things on that.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix deprecated sets.String

This is now available using generics.

Signed-off-by: Dirkjan Bussink <[email protected]>

* run make proto

Signed-off-by: Florent Poinsard <[email protected]>

* fix e2e test expectations

Signed-off-by: Florent Poinsard <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Florent Poinsard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants